5 tips for OSINT on the dark web

Michael James of OSINT Curious Project shares his advice on conducting OSINT on the dark web, including how to stay secure, anonymous and focused on your mission.

The OSINT community is growing. More organizations are dedicating professional roles to gathering open-source information to improve intelligence for a variety of use cases. And amateurs in the online sleuthing sphere have been gaining notoriety as they contribute to law enforcement investigations, understanding global conflicts, countering disinformation and more. Anyone on this OSINT spectrum can find helpful tips and resources with The OSINT Curious Project.

OSINT Curious provides free and tangible information to equip OSINT researchers with the right operational knowledge and safety tools to help them perform investigations safely and effectively. That’s why we asked one of their founding members, Michael James, to join us on the NeedleStack podcast. In this episode, James shares his insights on conducting OSINT on — and off — the dark web. Here are five key recommendations.

1. Think of the mission, not the tool

It’s easy to fixate on tools. The dark web itself — or the software like Tor to access it — is itself a tool. But first and foremost, researchers should focus on the mission. Often, a tool-first approach will yield lots of information, but little intelligence applicable to the mission, and understanding the difference between information and intelligence is critical.

Collecting information is an important early step (though not the first in the intelligence cycle) To get from raw information to a finished intelligence product, researchers must process and analyze the data to determine whether it’s relevant to the investigation and turn it into actionable insights. Which is why it’s essential to lead with the question of “why is this important,” rather than focus on which tools to use. 

(But if you don’t want some tool recommendations, we’ll see you down in the resources section.)

2. Don’t leave information on the table

Google is by far the most popular search engine, returning results from the vast expanse of the Internet. For a casual searcher, it’s often enough; but for those with specific OSINT objectives, they may be missing out. Checking other search engines like Bing, region-specific ones such as Yandex (Russia) or Baidu (China) can yield different results that may offer valuable context. Additionally, searching social media (on-platform or through purpose-built search tools) and the dark web may be appropriate for different investigations.

Read more: Leveraging the dark web in online investigations >

Getting what you need from the dark web can be tricky. First, you need to download and install a specialized browser, like Tor, or use one of the proxies which will allow you to browse the dark web via an ordinary browser (note: James does NOT recommend doing this, but it does work). Most importantly, you need to know what domain you are looking for, as dark web pages aren’t indexed as are surface web pages. There are keyword searches and indicators that researchers can use like Ahmia and Haystack, but for the most part, analysts must traverse the dark web without a clear map, taking extra precautions not to see something that can't later unsee or to stumble upon sites that aim to infect their endpoints and networks with malware.  

Learn how Silo for Research can give OSINT professionals access to the dark web without downloading Tor and with built-in security, anonymity and auditability >

3. Don’t be afraid of the dark web, but be smart about it

In spite of all the awful things on the dark web (in fact, because of them), the dark web can be a great information source for law enforcement, government and corporate investigations. Hear James talk about the types of investigations he frequently uses the dark web for and how that work yields real-world results. 

“That's one of the real benefits of OSINT for me, is taking a digital artifact and moving it into a physical world. It's the connection between the cyber and the real that really plays a big role in regard to personal security and privacy.”

— Michael James, S1E16 NeedleStack Podcast

As evidenced by the type of information to be found there, the dark web is a dangerous place. Researchers need to understand the risks and how to protect against them if and when they decide to venture into the dark web. Consider these questions:

• Is a dark web investigation necessary for your investigation? If the answer is no, stay away. If maybe or a yes, remember the tip about not leaving information on the table. Also consider, especially for hobbyists, you can’t unsee anything you find on the dark web.
• Am I protected against cybersecurity risks? Just clicking on a link or visiting a site could introduce malicious content to your machine and network. 
• Am I properly concealing my identify/affiliation? Contrary to popular belief, the dark web isn’t totally anonymous. Lots of details of your digital fingerprint are still passed to sites you visit, and they could be used to identify you or who you’re working for.
• Is my organization ready for me to access the dark web? For professionals accessing the dark web on behalf of their organization, make sure that proper policies have been put in place.

Get the basics on using the dark web in your investigation. Check our our blog series for everything you need to know >

4. Pivot, pivot, pivot: the OSINT tradecraft

OSINT researchers often spend days scraping information from different platforms, finding the identifying markers like WhatsApp numbers or Proton email accounts, comparing usernames on different sites, finding out which platforms a user is registered on, switching from social media sites to darknet marketplaces, backtracking to analyze historical views, and constantly scrutinizing their findings to see if it helps pinpoint the identity of a malicious actor. And that’s exactly what Michal James calls “the OSINT tradecraft” — being able to pivot from one piece of data, from one source to another. It’s key to compiling as much verified information as possible to help crack the case. 

Especially where the dark web is involved, researchers may find themselves pivoting between it and the surface web. Some dark web marketplaces are even mirrored on the surface web, but the surface-web versions can offer helpful information unavailable on the dark web; for example: surface websites may be “off the shelf” (e.g., WordPress, NginX, Apache) and expose new information. They may lack privacy restrictions, reveal server info status (what else is running on the server, ripe for URL jacking) and allow researchers to look for sitemaps through XML where authorship may be listed.

James walks through such an investigation full of twists and turns, where lots of pivot points culminated in identifying a dark web operator:

“It's the smoking gun all the time, but if you can layer more and more of these informational artifacts, then you can go through and continue to build a case.”

— Michael James, S1E16 NeedleStack Podcast

5. Biggest concern for investigators should be opsec

Operational security should always be the number one concern for all OSINT researchers, but especially the ones who are venturing out to the dark web. You don’t want to go to the Tor browser and log into your personal bank account; you always need to be aware that any of your actions can trigger scripts to deploy malware on your device. 

Researchers also need to be concerned about digital fingerprinting — anything you are looking at online can be looking back at you, and if an adversary figures out that they are being watched, they can compromise your investigation by covering their tracks or retaliating. Whether for a professional online investigator or a hobbyist, operational security should be top priority and using specialized tools and repeatable processes to separate their risky online activity from their daily work is essential for the success of their mission.

Resources and tool recommendations

James mentioned a number of useful resources that can help OSINT researchers in their investigations. You can find more tips and recommendations on the OSINT Curious website and in their Discord channel.

• Hunchly – OSINT software for automatically collecting, documenting and annotating every web page you visit
• Tails (for the hobbyist) – a portable operating system that protects against surveillance and censorship
• Echosec – OSINT tools for detecting potential threats with real-time data from mainstream social media, fringe networks, news outlets, and messaging apps
• Ahmia – anonymous network search engine
• Michael’s GitHub (useful tools for Google dorking searches through Katana, etc.)

Find Michael’s interview and more great guests on the NeedleStack podcast website. Or subscribe to the show to get episodes delivered straight to your inbox.